AFHRL-TR-90-46 


AIR  FORCE  SYSTEMS  COMMAND 
BROOKS  AIR  FORCE  BASE,  TEXAS  78235-5601 


NOTICE 


When  Government  drawings,  specifications,  or  other  data  are  used  for  any  purpose 
other  than  in  connection  with  a  definitely  Government-related  procurement,  the 
United  States  Government  incurs  no  responsibility  or  any  obligation  whatsoever. 
The  fact  that  the  Government  may  have  formulated  or  in  any  way  supplied  the  said 
drawings,  specifications,  or  other  data,  is  not  to  be  regarded  by  implication,  or 
otherwise  in  any  manner  construed,  as  licensing  the  holder,  or  any  other  person  or 
corporation;  or  as  conveying  any  rights  or  permission  to  manufacture,  use,  or  sell 
any  patented  invention  that  may  in  any  way  be  related  thereto. 

The  Public  Affairs  Office  has  reviewed  this  report,  and  it  is  releasable  to  the  ,.onal 
Technical  Information  Service,  where  it  will  be  available  to  the  general  public, 
including  foreign  nationals. 

This  report  has  been  reviewed  and  is  approved  for  publication. 


WILLIAM  E.  ALLEY,  Technical  Director 
Manpower  and  Personnel  Division 


HAROLD  G.  JENSEN,  Colonel,  USAF 
Commander 


REPORT  DOCUMENTATION  PAGE 

Form  Approved 

0MB  No.  0704-0188 

Public  reporting  burden  for  this  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  soi  ces. 
gathering  and  maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  This  burden  estimate  or  any  other  aspect  of  tn.s 
collection  of  information,  including  suggestions  for  reducing  this  burden, To  Washington  Heaaquarlers  Services,  Directorate  Tor  Information  Operations  and  Reports,  1215  Jefferson 
Davis  Highway,  Suite  1204,  Arlington,  VA  22202-4302.  and  to  the  Office  of  Management  and  Budget,  Paperwork  Reduction  Project  (0704-0188),  Washington.  DC  20503 

1.  AGENCY  USE  ONLY  (Leave  blank)  2.  REPORT  DATE  3.  REPORT  TYPE  AND  DATES  COVERED 

September  1990  Interim  Report  -  December  1989  -  May  1990 

4.  TITLE  AND  SUBTITLE 

Relationships  of  General  Ability,  Specific  Ability,  and  Job  Category  for 
Predicting  training  Performance 

5.  FUNDING  NUMBERS 

PE  -  62703F 

PR  -  7719 

TA  -  18 

WU  -  67 

6.  AUTHOR(S) 

Malcolm  James  Ree 

James  A.  Earles 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

Manpower  and  Personnel  Division 

Air  Force  Human  Resources  Laboratory 

Brooks  Air  Force  Base,  Texas  78235-5601 

8.  PERFORMING  ORGANIZATION 
REPORT  NUMBER 

AFHRL-TR-90-46 

9.  SPONSORING/MONITORING  AGENCY  NAMES(S)  AND  ADDRESS(ES) 

10.  SPONSORING/MONITORING  AGENCY 
REPORT  NUMBER 

11.  SUPPLEMENTARY  NOTES 

SC  Study  9657 

12a.  DISTRIBUTION/AVAILABIUTY  STATEMENT 

Approved  for  public  release;  distribution  is  unlimited. 

12b.  DISTRIBUTION  CODE 

13.  ABSTFIACT  (Maximum  200  words) 

Jr*  The  roles  of  general  ability  (g),  specific  abilities  (s),  and  job  category  are  investigated  as  predictors  of  final  school 
grade  for  78,041  Air  Force  enlistees  in  82  jobs.  General  cognitive  ability  and  specific  abilities  were  defined  by  scores 
on  the  first  and  subsequent  principal  components  of  the  enlistment  selection  and  classification  test,  the  Armed  Services 
Vocational  Aptitude  Battery.  A  linear  models  analysis  revealed  that  g  was  a  good  predictor  of  all  criterion  measures 
and  that  s  added  a  small  but  statistically  significant  amount.  Adjusting  the  final  school  grade  criterion  scale  using  job 
category  variables  significantly  improved  prediction.  \  ;  -  . ,  _ 

....  •  .  *  • 

.  r / '  - 

14.  SUBJECT  TERMS 

aptitude  test  psychometric  g 

linear  models  regression 

principal  components  validity 

15.  NUMBER  OF  PAGES 

16 

16.  PRICE  CODE 

17.  SECURITY  CLASSIFICATION  IS.  SECURITY  CLASSIFICATION  19.  SECURITY  CLASSIFICATION 
OF  REPORT  OF  THIS  PAGE  OF  ABSTRACT 

Unclassified  Unclassified  Unclassified 

20.  LIMITATION  OF  ABSTRACT 

UL 

Standard  Form  298  (Rev  2-89) 
Prescribed  by  ANS'  Std  Z3!*  1A 


NSN  754C -01  -280-5500 


SUMMARY 


Many  multiple-aptitude  test  batteries,  including  the  Armed  Services  Vocational  Aptitude  Battery 
(ASVAB),  used  for  assigning  or  classifying  individuals  to  jobs  or  for  occupational  counseling 
have  subtests  covering  a  broad  range  of  content  such  as  science,  mathematics,  reading, 
vocabulary,  clerical,  mechanical,  or  technical  knowledge.  This  content  reflects  a  belief  that 
performance  in  different  jobs  is  best  predicted  by  subtests  whose  content  appears  to  be  closely 
related  to  the  jobs.  It  has  been  demonstrated  that  the  subtests  of  a  multiple-aptitude  test 
battery  all  measure,  at  least  in  part,  an  examinee's  general  learning  ability  in  addition  to  the 
specific  abilities  implied  by  the  differing  contents  of  the  subtests. 

The  present  effort  investigated  the  utility  of  general  learning  ability  and  specific  abilities  for 
predicting  final  school  grades  in  82  Air  Force  technical  training  schools  Subjects  were  72.C41 
Air  rorCe  enlistees.  It  was  found  that  general  learning  ability  was  by  far  the  best  predictor  for 
the  training  grades;  however,  specific  abilities  improved  the  predictive  accuracy  by  a  small 
amount. 
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PREFACE 


This  effort  was  conducted  under  work  unit  77191867,  Research  on  Air  Force  Selection 
and  Classification.  The  authors  would  like  to  thank  SSgt  Steven  Hoffer,  AFHRL/SC,  for  his 
computer  analyses  Gratitude  is  extended  to  Ms  Jacobina  Skinner  for  assistance  with  the 
formulation  of  the  linear  models  and  the  counting  of  linearly  independent  vectors.  For  critical 
reading  of  the  urtexts  the  authors  thank  Linda  Sawin,  Thomas  Watson,  and  Lonnie  D. 
valentine.  Jr.,  all  of  AFHRL/MOA  and  Bill  Phalen  of  AFHRL/MOD.  William  Alley,  AFHRL/MO, 
is  owed  a  debt  of  gratitude  for  insightful  discussions  on  the  topic. 

This  effort  was  unique  as  it  contained  linear  models  with  over  900  vectors  and  as  such, 
was  computationally  cumbersome.  The  results  were  instructive  and  the  authors  believe 
Shakespeare  would  have  written  the  following  had  he  observed  the  results:  "Lord,  what  tools 
these  models  be. 1 
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RELATIONSHIPS  OF  GENERAL  ABILITY,  SPECIFIC  ABILITY,  AND  JOB 
CATEGORY  FOR  PREDICTING  TRAINING  PERFORMANCE 


I.  INTRODUCTION 

The  concept  of  general  cognitive  ability  or  psychometric  g  first  proposed  by  Galton  in  1883 
appeared  in  analyses  early  in  this  century  (Spearman,  1904).  It  has  been  and  remains  the 
center  of  much  controversy. 

Early  intelligence  test  developers  such  as  Binet  ano  Simon  were  proponents  of  g  but 
eventually  the  influence  of  multiple-ability  theorists  (Thurstone,  1938)  was  pervasive.  This  led 
to  the  development  of  multiple-aptitude  batteries.  The  Differential  Aptitude  Tests  (DAT),  the 
General  Aptitude  Test  Battery  (GATB),  and  the  Armed  Services  Vocational  Aptitude  Battery 
(ASVAB)  were  designed  to  measure  specific  abilities  and  to  make  specific  predictions  about 
employment  or  education.  Sets  of  test  scores  are  differentially  selected  or  differentially  weighted 
for  each  situation,  fulfilling  a  proposal  by  Hull  (1928).  The  different  composites  of  subtests 
used  by  the  military  for  job  placement  or  the  interpretation  of  score  profiles  in  counseling  are 
current  examples  of  the  application  of  multiple-ability  theory.  The  use  of  differential  weighting 
and  different  composites  led  to  multiple-aptitude  theory  being  termed  "a  theory  of  differential 
validity." 

Recently  the  primacy  of  g  as  a  predictor  has  again  become  the  subject  of  many  studies. 
The  December  1986  issue  of  Journal  of  Vocational  Behavior  (Gottfredson,  1986)  provided  much 
impetus  for  the  renewed  interest,  as  did  the  evidence  emerging  from  validity  generalization 
studies  (Hunter,  1983,  1984a,  1984b,  1984c;  Hunter,  Crosson,  &  Friedman,  1985). 

The  ASVAB  is  an  excellent  data  source  for  recent  studies  investigating  the  value  of  g  as 
a  predictor,  with  over  a  million  administrations  and  over  200,000  selections  to  job  training  each 
year.  Jones  (1988)  correlated  the  average  validity  of  the  ASVAB  subtests  for  predicting  training 
performance  with  the  g  saturation  of  the  subtests.  The  validities  were  the  correlations  between 
a  subject-weighted  (N  =  24,482)  average  over  37  Air  Force  jobs,  and  the  g  saturation  was 
measured  by  the  loadings  on  the  unrotated  first  principal  component  (see  Jensen,  1987).  She 
found  a  rank-order  correlation  of  .72,  demonstrating  a  strong  positive  relationship  between  g 
and  validity. 

Ree  and  Earles  (1990a)  investigated  the  predictive  utility  of  both  the  general  and  specific 
components  of  the  ASVAB  by  regressing  Air  Force  technical  school  grades  on  the  unrotated 
principal  component  scores  of  the  ASVAB  (g  was  represented  by  the  first  principal  component). 
Across  89  jobs  (sample  sizes  ranged  from  274  to  3,939  individuals),  the  average  correlation 
of  g  and  the  training  criterion  was  .764  corrected  for  range  restriction.  When  the  specific 
components  were  added  to  the  regressions,  the  increase  in  R^  averaged  .015. 

The  above  two  studies  examined  the  relationships  of  general  ability  and  specific  ability,  but 
neither  statistically  addressed  differences  in  job  category.  A  linear  models  analysis  allows  a 
statistical  investigation  of  the  relationships  among  these  variables.  It  also  allows  investigation 
of  the  effects  of  using  only  the  specific  components  (si...sn)  or  only  the  g  component,  as  well 
as  investigation  of  differences  in  the  mean  and  dispersion  of  grade  criteria  of  the  individual 
training  schools. 
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II.  METHOD 


Subjects 


The  subjects  were  78,041  nonprior-service  Air  Force  enlistees  who  had  tested  with  ASVAB 
parallel  Forms  11,  12,  or  13  from  1984  through  1988  and  who  had  completed  both  basic 
military  training  and  technical  training.  They  were  mostly  white,  male,  from  17  to  about  23 
years  old  and  high  school  graduates.  Table  1  shows  the  demographic  characteristics  of  the 
sample.  To  ensure  sufficient  statistical  power  (Kraemer,  1983)  for  within-job  regressions,  no 
job  with  fewer  than  274  cases  was  used.  This  is  the  same  sample  used  earlier  by  Ree  and 
Earles  (1990a). 


Table  i.  Educational  and  Demographic  Description  of  the  Sample 


Gender 

Proportion 

Age 

Proportion 

Male 

82.8 

17-18 

29.2 

Female 

17.2 

19-20 

37.7 

21-22 

18.8 

23  + 

14.3 

Ethnicity 

Education 

Black 

14.8 

Less  than  High  School 

.9 

Hispanic 

2.8 

High  School  Graduate 

79.8 

White 

80.3 

College  Experience 

6.1 

Other 

2.1 

College  Graduate 

1.3 

Other 

1.9 

Measures 


The  Armed  Services  Vocational  Aptitude  Battery  is  a  multiple-aptitude  test  battery  (DOD, 
1984)  composed  of  10  subtests,  as  shown  in  Table  2.  Except  for  the  Numerical  Operations 
and  Coding  Speed  subtests,  all  are  power  tests.  The  ASVAB  is  used  by  all  of  the  Armed 
Services  for  enlistment  qualification  and  initial  job  assignment.  It  is  normed  on  a  weighted, 
nationally  representative  sample  of  18-  to  23-year-old  youths  (Maier  &  Sims,  1986;  Wegner  & 
Ree,  1985).  The  battery  has  been  used  in  this  current  configuration  since  1980,  and  is  highly 
reliable  (Palmer,  Hartke,  Ree,  Welsh,  &  Valentine,  1988)  and  valid  (Wilbourn,  Valentine,  &  Ree, 
1984). 

There  are  three  generally  accepted  ways  of  estimating  the  g  component  of  a  set  of  variables 
(Jensen,  1980).  Ree  and  Earles  (1990b)  have  shown  for  the  ASVAB  that  estimates  of  g  from 
these  three  methods-principal  components,  principal  factors,  and  hierarchical  factor  analysis-all 
correlated  greater  than  .996.  Because  of  its  mathematical  simplicity  and  the  high  correlation 
among  the  various  g  estimates,  the  principal  components  methodology  was  chosen  to  represent 
g  and  specific  measures  of  the  ASVAB.  Table  3  gives  the  10  sets  of  ASVAB  principal 
component  score  weights  (Hotelling,  1933a,  1933b)  derived  previously  (Ree  &  Earles,  1990a). 

The  method  of  principal  components  is  a  procedure  for  forming  orthogonal  linear  composites 
of  observed  variables.  Kendall,  Stuart,  and  Ord  (1983)  noted  that  the  uncorrelated  principal 
component  scores  avoid  problems  of  colinearity  and  are  useful  for  regression  analyses 
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Table  2.  Subtests  of  the  ASVAB 


Number  of 

Subtest  items  Time 


General  Science  (GS) 

Arithmetic  Reasoning  (AR) 

Word  Knowledge  (WK) 

Paragraph  Comprehension  (PC) 
Numerical  Operations  (NO) 
Coding  Speed  (CS) 

Auto  and  Shop  Information  (AS) 
Mathematics  Knowledge  (MK) 
Mechanical  Comprehension  (MC) 
Electronics  Information  (El) 


25 

11 

30 

36 

35 

11 

15 

13 

50 

3 

84 

7 

25 

11 

25 

24 

25 

19 

20 

9 

Table  3.  Unrotated  Principal  Component  Weights  for  the  ASVAB  Subtests 


Principal  components 


1 

2 

3 

4 

5 

GS 

.13808 

-.11244 

-.21982 

-.29416 

19523 

AR 

.13715 

.03854 

-.39912 

.54694 

-.02066 

WK 

.13736 

.06649 

-.21381 

-.64261 

-  08976 

PC 

.12778 

.16656 

-.31273 

-.71570 

-.02359 

NO 

.11291 

.38342 

42663 

.23843 

-1.36760 

CS 

.09956 

.44464 

.75816 

.03679 

1  11560 

AS 

.10878 

-.43374 

.60474 

-.00918 

-  34001 

MK 

. 12965 

.12086 

-.bl  486 

t>4452 

.20353 

MC 

.12448 

-.30623 

.21087 

.39938 

.36281 

El 

.12857 

-.29635 

.14351 

-.13640 

-.00001 

6 

7 

8 

9 

10 

GS 

-.88893 

-1.05107 

.56764 

.46367 

-1.25618 

AR 

.26159 

.58641 

25640 

-1  51740 

-1  06178 

WK 

-.20343 

-.35471 

.19392 

-1.22910 

1.53259 

PC 

1.10958 

.48914 

-.18581 

.83254 

-.55741 

NO 

-.11449 

-.39672 

-.29306 

.20266 

-.11527 

CS 

-.14894 

.21734 

.13184 

-.06193 

-  04099 

AS 

.22086 

.62982 

1.28388 

.27471 

.26269 

MK 

-.26607 

.28551 

.29615 

1.16925 

1.09690 

MC 

.89768 

-1.19071 

-.72807 

-.02996 

.28081 

El 

-.78167 

.90823 

-1.43032 

.09391 

-.06884 

These  10  unrotated  principal  component  scores  were  the  measures  of  ability  under 
investigation.  The  first  unrotated  principal  component  served  as  a  measure  of  g  (Jensen. 
1980),  and  the  other  nine  as  the  measures  of  specific  ability  (si  to  S9). 

The  criteria  were  the  final  school  grades  (FSG)  received  by  the  subjects  in  each  of  the 
technica'  training  courses  (see  Ree  &  Earles,  1990a:  Wilbourn,  Valentine.  &  Ree,  1984).  In 
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iiilM  tet.hni.  dl  turning  schools,  the  FSG  is  the  average  of  four  fairly  short  multiple  choice 
tcctmii  al  knowledge  and  procedures  tests  However,  to  be  eligible  to  take  these  tests,  students 
ui  .st  first  pass  work  sample  tests  (frequently  called  performance  checks  )  In  most  technical 
tunning  schools,  these  performance  checks  may  be  repeated  numerous  times  until  the  subject 
: creeds  Although  some  subjects  are  removed  from  technical  training  for  failure  to  pass 
these  peitormance  checks,  no  easily  accessible  records  of  repeated  performance  check  scores 
exist  FSG  is  a  complex  criterion,  reflecting  more  than  printed  tests 

FSG  is  reported  as  a  numerical  grade  from  a  lowest  passing  of  70  to  a  highest  of  99. 
although  Same  schools  credit  60  as  passing  The  selectivity  and  difficulty  of  the  school  are 
not  reflected  in  the  criterion  measures  Each  criterion  is  scaled  independently  the  criterion 
means  and  standard  deviations  are  in  no  orderly  relation  to  one  another  This  tends  to  wrongly 
equate  a  grade  of  say,  85  in  each  school  Without  knowledge  of  differences  in  the  criteria, 
proper  inteipretation  is  difficult  and  potentially  misleading  For  example,  differences  in  the 
means  of  the  criteria  across  schools  may  lead  to  differences  in  the  regressions,  in  practice, 
this  adds  heterogeneity  to  the  criterion  and  the  accuracy  of  prediction  is  reduced 


Procedures 

Seven  linear  models  (Ward  &  Jennings,  1973)  were  constructed  and  tested  to  assess  the 
■contribution  of  g  and  St...S9,  and  to  adjust  for  job  category  in  predicting  the  criteria  The 
general  linear  model  underlies  all  linear  statistics,  such  as  regression,  analysis  of  variance, 
anonical  analysis,  factor  analysis,  and  discriminant  function  analysis  Testing  of  linear  models 
is  based  on  establishing  a  full  model  which  contains  as  much  information  as  is  ava'lable 
and  then  evaluating  the  loss  of  predictive  accuracy  resulting  from  the  elimination  of  portions 
of  that  information  ir.  simpler  models,  often  called  "restricted  models  The  restricted  models 
must  contain  variables  which  are  a  subset  of  the  variables  in  the  full  model  There  is  an  F 
statistic  associated  with  the  analysis  (see  Ward  &  Jennings,  1973,  p  67) 

Table  4  describes  the  linear  models  and  variables  used  in  the  present  investigation  The 
full  model  contained  902  linearly  independent  variables,  allowing  each  job  category  to  have  its 
own  inieicept  and  10  slopes,  one  for  each  of  the  10  principal  component  scores  There  were 
820  variables  associated  with  the  principal  components  (10  per  job  times  82  jobs)  and  82  job 
variables  (the  unit  vector  and  81  binaries-one  for  each  job.  less  one  to  eliminate  redundant 
vectors). 

Restricted  model  1  (RM1)  contained  the  unit  vector,  81  job  binaries,  and  82  interaction 
variables  (JPl)  to  allow  principal  component  1  (g)  to  have  its  own  relationship  (slope)  to  FSG 
for  each  job.  No  other  aptitude  information  was  included.  There  were  164  linearly  independent 
variables  in  this  model 

Restricted  model  2  (RM2)  contained  intercepts  for  each  job  (the  unit  vector,  all  but  one  of 
the  binaries  for  job  membership)  and  the  10  principal  component  scores  (Pi  to  P10),  for  a 
total  of  92  independent  predictors.  In  this  model  the  relationship  (slope)  between  each  aptitude 
predictor  and  FSG  was  constrained  to  be  the  same  for  all  jobs.  That  is,  g  had  a  single 
slope  across  all  jobs;  si  had  a  single  slope  across  all  jobs,  as  did  S2  and  so  on 

The  third  restricted  model  (RM3),  with  83  predictors,  had  the  unit  vector,  81  job  binaries 
(JOBS),  and  the  scores  on  principal  component  1  (g).  This  model  contained  all  the  job 
information  but  only  one  aptitude  predictor,  g,  which  was  constrained  to  have  the  same  slope 
across  all  jobs. 
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Table  4  Linear  Models  Used  in  the  Statistical  Tests 


FSG 


Full  Model 

Unit  vector  <  81  Jobs  binary  variables  -*  820  JP 1  to  JPlO  product  variables 


Restricted  Model  1 


FSG 

Unit  vector 

*  81 

Jobs 

binary 

Restricted 

Model 

2 

FSG 

Unit  vector 

i-  81 

Jobs 

binary 

Restricted 

Model 

3 

FSG 

Unit  vector 

t  81 

Jobs 

binary 

Restricted 

Model 

4 

FSG 

Unit  vector 

+  81 

Jobs 

binary 

Restricted 

Model 

5 

FSG 

Unit  Vector 

-r  PI 

to  P 

10 

variables  t  82  JP  1  product  variables 

variables  +  Pi  to  P10 

variables  +  Pi 

variables 


Restricted  Model  6 

FSG  Unit  vector  +  Pi 


Variables  used  in  me  linear  models 


Name 

FSG 
JOBS 
PI  to  P10 
JP  1  to  JP10 
Unit  Vector 


Description 

Final  School  Grade,  a  continuous  variable 

Categorical  variable,  1  if  in  job,  0  otherwise 

Scores  on  principal  components  1  to  10,  continuous  variables 

Product  variables  of  categorical  variable  JOBS  and  Pi  to  P10 

A  vector  with  every  element  equal  to  1 


The  fourth  restricted  model  (RM4)  of  82  independent  predictors  had  the  unit  vector  and 
81  job  binaries  Prediction  in  this  model  was  based  only  on  knowledge  of  job  category;  no 
information  on  aptitude  was  used 

The  fifth  restricted  model  (RM5),  with  11  linearly  independent  variables,  contained  the  unit 
vector  and  the  10  principal  component  scores  (Pi  to  P 1 0)  This  model  removed  all  job 
information  from  the  full  model,  leaving  only  aptitude  as  predictors 

Restricted  model  6  (RM6)  contained  only  two  linearly  independent  predictors,  principal 
component  1  (g)  and  the  unit  vector 

Each  statistical  test  compared  a  restricted  model  to  the  full  model  to  answer  questions 
about  the  relationships  of  ability  and  job  category  to  FSG.  Some  models  controlled  for  criterion 
scaling  and  allowed  estimation  of  criterion  difference  effects  All  statistical  tests  were  evaluated 
at  the  01  Type  I  error  rate 

While  the  linear  models  F  test  compares  the  difference  between  error  sums  of  squares  (or 
R2s),  the  relative  predictive  efficiency  of  the  models  was  evaluated  using  Rs  Brogden  (1946) 
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presents  a  proof  which  demonstrates  that  predictive  efficiency  is  linearly  and  directly  related 
to  R  (or  r  in  the  bivariate  case).  A  correlation  of  40  is  half  as  efficient  in  prediction  as  a 
correlation  of  80  The  predictive  efficiency  of  restricted  models  will  be  expressed  as  a  percent 
of  the  predictive  efficiency  of  the  full  model  in  all  but  one  instance. 


III.  RESULTS  AND  DISCUSSION 

Each  of  the  seven  linear  models  was  determined  to  be  significantly  different  from  zero 
Table  5  shows  the  correlauons  and  their  respective  F  statistics 


Table  5  Correlations,  Squared  Multiple  Correlations,  and  F  Tests  for  the  Models 


Model 

R 

R2 

dfi 

df2 

F 

Full 

62831 

.39477 

901 

77139 

55  77900 

RM1 

60812 

36981 

163 

77877 

280  37153 

R  M2 

.60863 

37042 

91 

77949 

503.98888 

RM3 

60334 

36402 

82 

77958 

544  17176 

RM4 

46419 

21547 

81 

77959 

264  34314 

RM5 

42814 

18330 

10 

78030 

1751.36070 

RM6 

.41803 

.17475 

1 

78039 

16525.06542 

Fach  restricted  model  was  tested  against  the  full  model,  and  all  of  their  R2s  were  found 
to  be  significantly  lower  The  standard  error  of  estimate  (Se)  oi  the  full  modeT  was  5  09  and 
those  for  the  restricted  models  were  5.16  or  greater. 

The  test  of  RM1  against  the  full  model  yielded  a  statistically  significant  but  small  difference 
(F  4.31,  dfi  =  731,  df2  =  77139).  There  was  an  increase  of  .08  in  the  Se  The  R 
difference  of  02019  (.62831  -  .60812  =.02019)  represents  a  loss  in  relative  predictive  efficiency 
of  3%  (  02019/  62831  =  .032  3).  This  restricted  model  contains  all  job  information  (the  job 
binaries)  and  variables  to  allow  g  to  have  a  different  slope  for  each  job.  No  information  on 
specific  ability  is  included  and  the  loss  in  relative  predictive  efficiency  was  very  small. 

The  second  restricted  model  (RM2)  tested  against  the  full  model  showed  a  statistically 
significant  difference  (F  =  3.83,  dfi  =  810,  df2  =  77139),  but  a  relatively  small  R  difference 
of  01968  (a  loss  of  3%  relative  predictive  efficiency)  and  a  commensurately  small  increase  in 
Se  from  5.09  to  5.16.  RM2  creates  parallel  regression  lines  for  each  principal  component 
across  jobs,  but  allows  each  of  the  10  principal  components  to  have  its  own  slope.  For 
example,  the  slope  for  principal  component  1  is  the  same  for  each  job  and  the  slope  for 
principal  component  2  is  the  same  for  each  job,  but  the  slopes  of  principal  components  1  and 
2  are  not  necessarily  the  same.  This  model  specifies  that  the  predictive  value  of  aptitude  is 
the  same  for  each  job,  differing  only  in  level. 

The  test  of  RM3  versus  the  full  model  determines  if  constraining  g  to  a  single  slope  across 
all  jobs,  removing  specific  abilities,  and  retaining  job  information  (job  binaries)  is  less  predictive. 
This  yielded  a  significant  difference  (F  =  4.78,  dfi  =  819,  df2  =  77139)  in  R2  The  Se 
increased  by  .10.  There  was  a  small  loss  in  relative  predictive  efficiency,  4%,  for  giving  up 
all  the  specific  ability  information  and  the  individual  slopes  on  g. 


6 


The  test  of  RM4  (which  contains  only  the  job  binaries)  against  the  full  model  allows 
estimation  of  the  importance  of  aptitude  The  statistical  test  found  the  models  to  be  significantly 
different  (F  =  27.86,  df i  =  820,  df2  =  77139).  The  R  difference  was  .16412,  with  a  loss  of 
relative  predictive  efficiency  of  26%  and  an  Se  increase  of  .67.  Giving  up  information  on 
aptitude  was  costly. 

Furthermore,  RM4  has  a  predictive  efficiency  of  46%  relative  to  perfect  prediction.  The 
criterion  variable  differed,  at  least  to  scale  mean,  for  the  jobs  investigated.  These  differences 
added  heterogeneity  to  the  criterion.  Interpretation  of  the  regressions  without  consideration  of 
the  differences  in  the  criterion  would  be  misleading. 

The  next  linear  models  test,  comparing  RM5  to  the  full  model,  allows  each  principal 
component  to  have  a  separate  slope;  but  any  given  principal  component  has  the  same  slope 
across  all  jobs.  RMf  eliminates  all  information  on  jobs  and  retains  only  constrained  information 
on  the  aptitude  predictors.  There  was  a  statistically  significant  difference  (F  =  30.25,  dfi  = 
891,  df2=  77139)  between  the  models.  The  R  difference  was  .20017,  a  large  decrease  with 
a  loss  in  predictive  efficiency  of  32%.  Additionally,  the  increase  in  Se  was  from  5  09  to  5.88 
from  the  full  to  the  restricted  model.  Clearly,  giving  up  information  about  jobs  was  costly. 

The  test  of  RM6  against  the  full  model  asks  if  g  is  as  good  a  predictor  as  all  the  information 
in  the  full  model.  The  F  statistic  (F  =31.15,  dfi  =  900,  dfe  =  77139)  showed  a  significant 
difference  between  the  models.  There  was  a  .21028  decrease  in  the  R  (a  33%  loss  in  relative 
predictive  efficiency)  and  a  substantial  increase  of  .82  (5.91  -  5.09)  in  the  Se 

Finally,  a  test  of  the  difference  between  RM1  and  RM3  was  conducted  to  determine  the 
statistical  difference  and  the  relative  predictive  efficiency  loss  for  using  a  single  slope  for  g 
as  opposed  to  using  a  different  slope  for  each  job.  It  was  statistically  significant  (F  =  8.83, 
dfi  =  81 ,  df2  =  77877).  There  was  an  R  difference  of  .00478  and  a  .02  Se  increase.  Although 
there  was  not  the  same  relationship  between  g  and  each  job,  the  differences  among  these 
relationships  were  small.  The  loss  in  relative  predictive  efficiency  was  less  than  1%. 


IV.  CONCLUSIONS 

These  analyses  disclosed  several  interesting  results.  First,  all  the  principal  components 
were  useful  in  predicting  the  criteria,  much  as  was  found  earlier  (Ree  &  Earles,  1990a).  In 
the  ASVAB,  the  specific  abilities,  as  represented  by  the  second  through  tenth  principal 
components,  added  to  the  accuracy  of  prediction,  but  only  a  small  amount. 

The  results  also  indicated  that  interpretation  of  regressions  without  reference  to  job  categories 
could  be  very  misleading.  It  should  be  noted  that  a  little  more  than  two-thirds  of  the  predictive 
efficiency  of  the  full  model  came  from  the  knowledge  of  job  category. 

Controlling  job  category  information,  the  role  of  specific  aptitude  information  accounted  for 
an  increase  of  about  1  (RM2  versus  RM3)  to  3  (full  model  versus  RM1)  percent  in  relative 
predictive  efficiency.  When  only  job  categories  and  g  were  used,  the  consequence  of  using 
a  single  slope  for  g  instead  of  a  separate  slope  for  g  for  each  job  (RM1  versus  RM3)  was 
very  small. 

In  the  statistical  process  of  aggregating  correlations  to  describe  relationships  across  jobs, 
accounting  for  job  category  was  important.  General  cognitive  ability  (g)  has  been  shown  to  be 
a  good  predictor  for  all  of  the  jobs.  The  addition  of  measures  of  specific  ability  increased 
predictive  efficiency  but  only  a  small  amount.  Personnel  selection  systems  with  few  applicants 
would  not  be  likely  to  detect  the  statistical  effects  of  such  small  contributions  of  specific 
abilities. 
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Finally,  consideration  of  the  need  for  job-specific  slopes  for  g  indicated  that  the  use  of  a 
common  slope,  though  not  optimum,  is  not  very  costly. 

The  subtests  of  the  ASVAB  all  contribute  about  equally  to  the  measurement  of  g,  but  their 
non-g  contributions  are  small  and  unequal  (Ree  &  Earles,  1990b).  The  present  study 
demonstrates  the  utility  of  g  and  the  relative  lack  of  utility  of  S1...S9  as  predictors  of  FSG  It 
follows  that  any  attempt  to  aggregate  the  ASVAB  subtests  into  equally  reliable  composites  for 
the  purpose  of  creating  differential  prediction  of  FSG  can  yield  a  small  amount  of  improvement 
Additional  cognitive  or  non-cognitive  measures  outside  of  the  ASVAB  will  be  required  to  produce 
such  differential  prediction. 

Because  FSG  is  not  the  only  criterion  of  importance,  this  study  should  be  replicated  using 
job  performance  measures,  supervisory  ratings,  and  other  criteria. 
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